Annan and Hargreaves : Reliability of Cmip

We consider paradigms for interpretation and analysis of the CMIP3 en-2 semble of climate model simulations. The dominant paradigm in climate sci-3 ence, of an ensemble sampled from a distribution centred on the truth, is con-4 trasted with the paradigm of a statistically indistinguishable ensemble, which 5 has been more commonly adopted in other fields. This latter interpretation 6 (which gives rise to a natural probabilistic interpretation of ensemble out-7 put) leads to new insights about the evaluation of ensemble performance. Us-8 ing the well-known rank histogram method of analysis, we find that the CMIP3 9 ensemble generally provides a rather good sample under the statistically in-10 distinguishable paradigm, although it appears marginally over-dispersive and 11 exhibits some modest biases. These results contrast strongly with the incom-12 patibility of the ensemble with the truth-centred paradigm. Thus, our anal-13 ysis provides for the first time a sound theoretical foundation, with empir-14 ical support, for the probabilistic use of multi-model ensembles in climate 15 research.